Enhancing Wind Power Forecast Precision via Multi-head Attention Transformer: An Investigation on Single-step and Multi-step Forecasting

The main objective of this study is to propose an enhanced wind power forecasting (EWPF) transformer model for handling power grid operations and boosting power market competition. It helps reliable large-scale integration of wind power relies in large part on accurate wind power forecasting (WPF). The proposed model is evaluated for single-step and multi-step WPF, and compared with gated recurrent unit (GRU) and long short-term memory (LSTM) models on a wind power dataset. The results of the study indicate that the proposed EWPF transformer model outperforms conventional recurrent neural network (RNN) models in terms of time-series forecasting accuracy. In particular, the results reveal a minimum performance improvement of 5% and a maximum of 20% compared to LSTM and GRU. These results indicate that the EWPF transformer model provides a promising alternative for wind power forecasting and has the potential to significantly improve the precision of WPF. The findings of this study have implications for energy producers and researchers in the field of WPF.Comment: This paper is accepted in IJCNN2

Unlocking the capabilities of explainable fewshot learning in remote sensing

Recent advancements have significantly improved the efficiency and effectiveness of deep learning methods for imagebased remote sensing tasks. However, the requirement for large amounts of labeled data can limit the applicability of deep neural networks to existing remote sensing datasets. To overcome this challenge, fewshot learning has emerged as a valuable approach for enabling learning with limited data. While previous research has evaluated the effectiveness of fewshot learning methods on satellite based datasets, little attention has been paid to exploring the applications of these methods to datasets obtained from UAVs, which are increasingly used in remote sensing studies. In this review, we provide an up to date overview of both existing and newly proposed fewshot classification techniques, along with appropriate datasets that are used for both satellite based and UAV based data. Our systematic approach demonstrates that fewshot learning can effectively adapt to the broader and more diverse perspectives that UAVbased platforms can provide. We also evaluate some SOTA fewshot approaches on a UAV disaster scene classification dataset, yielding promising results. We emphasize the importance of integrating XAI techniques like attention maps and prototype analysis to increase the transparency, accountability, and trustworthiness of fewshot models for remote sensing. Key challenges and future research directions are identified, including tailored fewshot methods for UAVs, extending to unseen tasks like segmentation, and developing optimized XAI techniques suited for fewshot remote sensing problems. This review aims to provide researchers and practitioners with an improved understanding of fewshot learnings capabilities and limitations in remote sensing, while highlighting open problems to guide future progress in efficient, reliable, and interpretable fewshot methods.Comment: Under review, once the paper is accepted, the copyright will be transferred to the corresponding journa

WATT-EffNet: A Lightweight and Accurate Model for Classifying Aerial Disaster Images

Incorporating deep learning (DL) classification models into unmanned aerial vehicles (UAVs) can significantly augment search-and-rescue operations and disaster management efforts. In such critical situations, the UAV's ability to promptly comprehend the crisis and optimally utilize its limited power and processing resources to narrow down search areas is crucial. Therefore, developing an efficient and lightweight method for scene classification is of utmost importance. However, current approaches tend to prioritize accuracy on benchmark datasets at the expense of computational efficiency. To address this shortcoming, we introduce the Wider ATTENTION EfficientNet (WATT-EffNet), a novel method that achieves higher accuracy with a more lightweight architecture compared to the baseline EfficientNet. The WATT-EffNet leverages width-wise incremental feature modules and attention mechanisms over width-wise features to ensure the network structure remains lightweight. We evaluate our method on a UAV-based aerial disaster image classification dataset and demonstrate that it outperforms the baseline by up to 15 times in terms of classification accuracy and $38.3\%$ in terms of computing efficiency as measured by Floating Point Operations per second (FLOPs). Additionally, we conduct an ablation study to investigate the effect of varying the width of WATT-EffNet on accuracy and computational efficiency. Our code is available at \url{https://github.com/TanmDL/WATT-EffNet}.Comment: This paper is accepted in IEEE Trans. GRS

Developing Generative Adversarial Networks for Classification and Clustering: Overcoming Class Imbalance and Catastrophic Forgetting

Generative adversarial networks (GAN) have attracted significant attention from the research community due to their superior clustering and classification abilities. However, data imbalance and continual learning settings cause poor GAN’s performance. In class imbalance problems, the thesis focuses on GAN-based novel data augmentation and oversampling strategies. In GAN-based augmentation, a three-player adversarial GAN game method called class-dependent mixture generator spectral GAN (MGSGAN) is introduced. MGSGAN plays an adversarial game with a discriminator and hyper-spectral classifier to improve its performance by considering the actual data and class-conditionals augmentation generated samples. Our experiments revealed that MGSGAN outperforms other leading approaches by a significant margin for hyperspectral image classification. A second method relies on oversampling techniques, where two adversarial oversampling strategies (adversarial oversampling and adversarial data space oversampling) are introduced with a mixtures of generator parameters updated through classifier parameters. A novel latent preserving single generator under the proposed three-player GAN game settings is introduced. We present four strategies covering all possible GAN games that can be played under three-player settings. The methods were competitive to the state-of-the-art GAN-based oversampling methods. GAN-based adversarial continual learning (ACL) is a recently developed iterative process where a shared feature network is played in an adversarial game with task-specific discriminators. Each task-specific network is introduced to handle the catastrophic forgetting of past tasks. As the job grows, the network structure of ACL also grows simultaneously. Therefore, ACL is not scalable. We propose an online Scalable Adversarial Continual Learning (SCALE) framework where the parameter generator transforms shared features into task-specific features. SCALE surpasses popular baseline methods on accuracy and execution time. Finally, in conjunction with a clustering inference network, a uniform prior-based conditional generative model, ClusterGAN, has recently achieved sub-optimal clustering performance. ClusterGAN may fail to generate all the modes due to disentanglement in the clustered inference network that can’t be fully achieved through only generative. SIMI-ClusterGAN is a new method where a prior network is introduced together with three additional losses to increase disentanglement in the clustering inference network. The method has been validated through benchmark datasets in which balanced and imbalanced scenarios have been considered

WATT-EffNet: a lightweight and accurate model for classifying aerial disaster images

Incorporating deep learning (DL) classification models into unmanned aerial vehicles (UAVs) can significantly augment search-and-rescue operations and disaster management efforts. In such critical situations, the UAV's ability to promptly comprehend the crisis and optimally utilize its limited power and processing resources to narrow down search areas is crucial. Therefore, developing an efficient and lightweight method for scene classification is of utmost importance. However, current approaches tend to prioritize accuracy on benchmark datasets at the expense of computational efficiency. To address this shortcoming, we introduce the Wider ATTENTION EfficientNet (WATT-EffNet), a novel method that achieves higher accuracy with a more lightweight architecture compared to the baseline EfficientNet. The WATT-EffNet leverages width-wise incremental feature modules and attention mechanisms over width-wise features to ensure the network structure remains lightweight. We evaluate our method on a UAV-based aerial disaster image classification dataset and demonstrate that it outperforms the baseline by up to 15 times in terms of classification accuracy and 38.3% in terms of computing efficiency as measured by Floating Point Operations per second (FLOPs). Additionally, we conduct an ablation study to investigate the effect of varying the width of WATT-EffNet on accuracy and computational efficiency. Our code is available at \url{https://github.com/TanmDL/WATT-EffNet}.Civil Aviation Authority of Singapore (CAAS)Nanyang Technological UniversityThis work was supported by the Civil Aviation Authority of Singapore and Nanyang Technological University (NTU) in collaboration with the Air Traffic Management Research Institute